Image Restoration Research Articles

Image enhancement and blur pixel identification with optimization-enabled deep learning for image restoration

Image enhancement and blur pixel identification with optimization-enabled deep learning for image restoration

Some Conditions and Perturbation Theorem of Irregular Wavelet/Gabor Frames in Sobolev Space

Due to its potential applications in image restoration and deep convolutional neural networks, the study of irregular frames has interested some researchers. This paper addresses irregular wavelet systems (IWSs) and irregular Gabor systems (IGSs) in Sobolev space HsR. We obtain the sufficient and necessary conditions for IWS and IGS to be frames. By applying these conditions, we also derive the characterizations of IWS and IGS to be frames. Finally, we discuss the perturbation theorem of irregular wavelet frames (IWFs) and irregular Gabor frames (IGFs). We also provided some examples to support our results.

Group sparsity residual constraint model with weighted log-sum penalty for image restoration

In the field of image restoration, non-convex penalties and non-local self-similarity (NSS) prior have been found to enhance image edges and textures. However, NSS-based methods often overlook the inherent variations within patch groups by treating them as a single entity for processing. Despite the existence of numerous convex or non-convex regularizers, few of them have been able to adaptively adjust penalty strength based on NSS information in order to preserve fine details in an image. To address this issue, we propose a novel group sparsity residual constraint model with weighted log-sum penalty (GSRC-Log) for image restoration in this paper. The proposed regularization term is parameterized by an adaptive scalar parameter that relates to NSS, allowing the model to prioritize high NSS group sparse coefficients while penalizing low NSS coefficients. For image denoising, we provide detailed analysis on the optimal condition of our proposed model and prove that it has a simple closed-form global solution. For image restoration tasks, we utilize the ADMM algorithm to solve the GSRC-Log problem and empirically verify its convergence. Extensive experimental results on both image denoising and compressive sensing (CS) recovery demonstrate that our proposed GSRC-Log outperforms many popular or state-of-the-art methods.

SGNet: Efficient Snow Removal Deep Network with a Global Windowing Transformer

Image restoration under adverse weather conditions poses a challenging task. Previous research efforts have predominantly focused on eliminating rain and fog phenomena from images. However, snow, being another common atmospheric occurrence, also significantly impacts advanced computer vision tasks such as object detection and semantic segmentation. Recently, there has been a surge of methods specifically targeting snow removal, with the majority employing visual Transformers as the backbone network to enhance restoration effectiveness. Nevertheless, due to the quadratic computations required by Transformers to model long-range dependencies, this significantly escalates the time and space consumption of deep learning models. To address this issue, this paper proposes an efficient snow removal Transformer with a global windowing network (SGNet). This method forgoes the localized windowing strategy of previous visual Transformers, opting instead to partition the image into multiple low-resolution subimages containing global information using wavelet sampling, thereby ensuring higher performance while reducing computational overhead. Extensive experimentation demonstrates that our approach achieves outstanding performance across a wide range of benchmark datasets and can rival methods employing CNNs in terms of computational cost.

Efficient image restoration via non-convex total variation regularization and ADMM optimization

This article presents a novel approach to image restoration utilizing a unique non-convex l1/2-TV regularization model. This model integrates the l1/2-quasi norm as a regularization function, introducing non-convexity to promote sparsity and unevenly penalize elements, thereby enhancing restoration outcomes. To tackle this model, an efficient algorithm named the Alternating Direction Method of Multipliers, based on the Lagrangian multiplier, is introduced. This effectively prevents the penalty parameter from reaching infinity and ensures excellent convergence behavior. The proposed algorithm decomposes the optimization problem into subproblems, for which closed-form solutions are derived, particularly addressing the challenging l1/2 regularization problem. To validate its effectiveness, a comprehensive set of experiments are conducted to compare its performance with existing methods. The experimental results demonstrate that the proposed model performs well in both qualitative and quantitative evaluations. Consequently, the proposed model is not only efficient and stable but also exhibits excellent convergence behavior.

Learning the Hybrid Nonlocal Self-Similarity Prior for Image Restoration

As an immensely important characteristic of natural images, the nonlocal self-similarity (NSS) prior has demonstrated great promise in a variety of inverse problems. Unfortunately, most current methods utilize either the internal or the external NSS prior learned from the degraded image or training images. The former is inevitably disturbed by degradation, while the latter is not adapted to the image to be restored. To mitigate such problems, this work proposes to learn a hybrid NSS prior from both internal images and external training images and employs it in image restoration tasks. To achieve our aims, we first learn internal and external NSS priors from the measured image and high-quality image sets, respectively. Then, with the learned priors, an efficient method, involving only singular value decomposition (SVD) and a simple weighting method, is developed to learn the HNSS prior for patch groups. Subsequently, taking the learned HNSS prior as the dictionary, we formulate a structural sparse representation model with adaptive regularization parameters called HNSS-SSR for image restoration, and a general and efficient image restoration algorithm is developed via an alternating minimization strategy. The experimental results indicate that the proposed HNSS-SSR-based restoration method exceeds many existing competition algorithms in PSNR and SSIM values.

Bidirectional image denoising with blurred image feature

Image denoising remains a classical yet still challenging problem, because the noise can destroy details and cause severe information loss. In recent years, various well-designed CNN-based methods have been extensively applied in image denoising because of the strong learning ability. However, most of them share an unidirectional procedure, which directly learns a mapping from noisy input to a clean image without focusing on the over-smoothed state of the denoising process, limiting the richness of extracted features. Different from previous works, we propose a blurred image feature guided CNN (BFCNN) network that implements a novel blurring-adjusting strategy to address the complex denoising problem via two stages. In stage 1, we build a blurring module (BM) to capture over-smoothed features from noisy observations and generate the blurred image restoration, which is a less informative version of the clean image. Furthermore, a multi-level concatenating module (CM) and an adjusting module (AM) are then designed to recover more detailed information in stage 2. These two modules are jointly designed to restore a properly-smoothed image from the over-smoothed blurred image and the given under-smoothed noisy image. Comparing to the traditional denoising process, the proposed blurring-adjusting strategy produces a precise denoised image more efficiently by converting the unidirectional denoising process into a bidirectional denoising process. To our knowledge, this is the first study that utilizes the over-smoothed image to address the denoising problem. Extensive experiments demonstrate the superiority of our BFCNN with more accurate reconstruction quality and achieve competitive quantitative results among current CNN-based methods. This research will release the code upon acceptance.

RobustFace: a novel image restoration technique for face adversarial robustness improvement

RobustFace: a novel image restoration technique for face adversarial robustness improvement

GANSharp: High-definition image reconstruction using generative adversarial networks

In the realm of digital imaging, enhancing low-resolution images to high-definition quality is a pivotal challenge, particularly crucial for applications in medical imaging, security, and remote sensing. Traditional methods, primarily relying on basic interpolation techniques, often result in images that lack detail and fidelity. GANSharp introduces an innovative GAN-based framework that substantially improves the generator network, incorporating adversarial and perceptual loss functions for enhanced image reconstruction. The core issue addressed is the loss of critical information during down-sampling processes. To counteract this, we proposed a GAN-based method leveraging deep learning algorithms, trained using sets of both low- and high-resolution images. Our approach, which focuses on expanding the generator network’s size and depth and integrating adversarial and perceptual loss, was thoroughly evaluated on various benchmark datasets. The experimental results showed remarkable outcomes. On the Set5 dataset, our method achieved a PSNR of 34.18 dB and a SSIM of 0.956. Comparatively, on the Set14 dataset, it yielded a PSNR of 31.16 dB and an SSIM of 0.920, and on the B100 dataset, it achieved a PSNR of 30.51 dB and an SSIM of 0.912. These results were superior or comparable to those of existing advanced algorithms, demonstrating the proposed method’s potential in generating high-quality, high-resolution images. Our research underscores the potency of GANs in image super-resolution, making it a promising tool for applications spanning medical diagnostics, security systems, and remote sensing. Future exploration could extend to the utilization of alternative loss functions and novel training techniques, aiming to further refine the efficacy of GAN-based image restoration algorithms.

Nonlinear Parabolic Double Phase Variable Exponent Systems with Applications in Image Noise Removal

In this paper, a novel parabolic system involving nonlinear and nonhomogeneous partial differential operator with variable growth structure is introduced for investigating the image denoising and restoration. More precisely, our model is based on regularizing the classical models involving variable exponent operators by considering a nonlinear operator with double phase flux. We begin by investigating theoretically the solvability to the parabolic system under consideration. Under the setting of Musielak-Orlicz spaces, we build a suitable functional framework to study the proposed system. Therefore, we develop the Faedo-Galerkin approach to demonstrate the existence and uniqueness of a weak solution to our model. To illustrate our theoretical results in the context of image noise removal, we present various numerical implementations on some grayscale images. To enrich these simulations, we test the robustness efficiency of the proposed model in the so-called Magnetic Resonance Images (MRI). The obtained numerical results claim that our model is more efficient and robust against noise, in comparison (visually and quantitatively) to some existing state-of-the-art methods.

An intelligent lithology recognition system for continental shale by using digital coring images and convolutional neural networks

Lithology identification is a critical component of reservoir evaluation and hydrocarbon development. However, the continental shale rocks, characterized by complex lithologies and developed laminations, present new challenges to traditional lithology identification techniques. In this study, we develop a centimeter-level intelligent identification system for continental shale lithology, leveraging digital image preprocessing and deep learning methods. Specifically, we construct a dataset from 400 meters of drilling cores, encompassing 24 types of lithology and 152 lithological transitions. The downsampling and oversampling methods are employed to balance the dataset. The digital image denoising and restoration techniques are then utilized to enhance rock image features. Different neural network models are examined to obtain the optimum predictive results. The results indicate that data balancing improves the TOP1 and TOP5 accuracies by 10% and 5%, respectively. Gaussian low-pass filter denoising, in comparison to other methods, increases the TOP1 and TOP5 accuracies by 2%, establishing it as the superior denoising technique for lithology recognition. Considering both computational efficiency and prediction accuracy, EfficientNet emerges as the most effective model for continental shale image recognition. The optimized model attains up to 60% TOP1 and 95% TOP5 across 22 categories in drilling core samples. This study delves into the intelligent identification of macroscopic drilling core images of continental shale, and the findings can offer technical support for the interpretation and evaluation of continental shale reservoirs.

VCAFusion: An infrared and visible image fusion network with visual perception and cross-scale attention

Infrared and visible image fusion methods aim to combine salient target instances and abundant texture details into fused images. However, due to the interference of harsh conditions, such as dense smoke, fog, and intense light, it is feasible for external interference information to be integrated into the fused image, which seriously affects the image quality. To this end, we propose an infrared and visible image fusion network with visual perception and cross-scale attention modules, termed VCAFusion, which integrates critical information from source images in harsh conditions more efficiently. Specifically, considering that the human eye can identify key information under adverse conditions, we design a visual perception module (VPM), guiding information integration from the perspective of human visual perception. In addition, we propose a cross-scale attention module (CSAM) based on shifted window cross-attention, which aims to capture the long-distance correlation between adjacent scale features, providing more accurate image information for image restoration. Experiments with a variety of datasets reveal that the VCAFusion can adaptively retain image information and further improve image generation ability in harsh conditions.

MuLA-GAN: Multi-level attention GAN for enhanced underwater visibility

The underwater environment presents unique challenges (color distortions, reduced contrast, blurriness) hindering accurate analysis. This work introduces MuLA-GAN, a novel approach leveraging Generative Adversarial Networks (GANs) and specifically adapted Multi-Level Attention for comprehensive underwater image enhancement. MuLA-GAN integrates Multi-Level Attention within the GAN architecture to prioritize learning discriminative features crucial for precise image restoration. These relevant features encompass information on local details within image regions leveraged by spatial attention and features at various scales across the entire image captured by multi-level attention. This allows MuLA-GAN to identify and enhance objects, textures, and edges obscured by underwater distortions while also reconstructing a more accurate and visually clear representation of the underwater scene by analyzing low-level information like edges and textures, as well as high-level information like object shapes and global scene information. By selectively focusing on these relevant features, MuLA-GAN excels at capturing and preserving intricate details in underwater imagery, which is essential for various marine research, exploration, and resource management applications. Extensive evaluations on diverse datasets (UIEB test, UIEB challenge, U45, UCCS) demonstrate MuLA-GAN's superior performance compared to existing methods. Additionally, a specialized bio-fouling and aquaculture dataset confirms the model's robustness in challenging environments. On the UIEB test dataset, MuLA-GAN achieves exceptional Peak Signal-to-Noise Ratio (PSNR) (25.59) and Structural Similarity Index (SSIM) (0.893) scores, surpassing Water-Net (24.36 PSNR, 0.885 SSIM). This work addresses a significant research gap in underwater image enhancement by demonstrating the effectiveness of combining GANs with specifically adapted Multi-Level Attention mechanisms. This tailored approach offers a novel and comprehensive framework for restoring underwater image quality, providing valuable insights for accurate underwater scene analysis. The source code for MuLA-GAN is publicly available on GitHub at https://github.com/AhsanBaidar/MuLA_GAN.git

Virtual cleaning of sooty murals in ancient temples using twice colour attenuation prior

Murals are artworks painted on walls that represent various cultures, traditions, historical periods, spiritual narratives and civilisations. Unfortunately, before modern preventive measures were implemented, murals suffered considerable degradation from accumulating substances, such as incense oil and carbon, leading to a darkening or blackening phenomenon called soot. The soot severely and extensively damaged the murals’ colour and motifs. However, no highly effective remedy is available to restore them to their original state. A novel approach for virtual cleaning of this degradation uses hyperspectral and image enhancement technologies. Existing techniques for sooty mural image restoration primarily use dark channel prior and Retinex bilateral filtering. During this process, the dark channel prior estimates the concentration of soot and removes it, restoring the mural’s colour with Retinex bilateral filtering. However, this restoration method has issues, such as neglecting the difference between soot and haze, resulting in a generally darker mural image. This result requires brightness adjustment, and the method presents further challenges, such as texture loss, colour distortion in some areas and a reduction in clarity. Therefore, we propose a new technique that integrates an inverse operation and twice colour attenuation prior to cleaning sooty murals. This technique involves normalised cross-correlation to match and stitch images to the different synthesised bands to reconstruct the mural image to be processed, which removes black marks from areas of mural loss. The sooty mural then undergoes an inverse operation, and the colour attenuation prior is used on the inverse image. The processed mural is then inverted again, transforming the colour of soot to white. Finally, the colour attenuation prior is again applied to the colour-changing mural, enabling virtual cleaning of the sooty murals. The results confirm the enhanced adaptability of our novel approach for the virtual cleaning of sooty murals. Moreover, the proposed method exhibits superior performance for various performance parameters.

Scientific mapping and bibliometric analysis of research advancements in underwater image enhancement

Underwater Image Enhancement (UIE) addresses challenges in marine resource exploitation such as absorption, noise, and low contrast. This paper employs bibliometric methods and data from Web of Science to analyze UIE literature. The analysis reveals that research on UIE entered a rapid development stage starting from 2017. Three main areas of technical support for research: underwater image enhancement methods, deep learning-based methods, and underwater image restoration and depth estimation. Hotspots in research include enhancement, deep learning, restoration, feature extraction, and quality assessment. Drawing from the bibliometric analysis, this paper proposes a conceptual framework for UIE research and presents five research suggestions: enhancing method robustness and adaptability, improving real-time performance, integrating software and hardware, establishing underwater image benchmark datasets, and refining quality evaluation systems for underwater images. This study offers valuable references and guidance for the future exploration and development of the UIE field.

Deep M2CDL: Deep Multi-Scale Multi-Modal Convolutional Dictionary Learning Network.

For multi-modal image processing, network interpretability is essential due to the complicated dependency across modalities. Recently, a promising research direction for interpretable network is to incorporate dictionary learning into deep learning through unfolding strategy. However, the existing multi-modal dictionary learning models are both single-layer and single-scale, which restricts the representation ability. In this paper, we first introduce a multi-scale multi-modal convolutional dictionary learning ( M2CDL) model, which is performed in a multi-layer strategy, to associate different image modalities in a coarse-to-fine manner. Then, we propose a unified framework namely Deep M2CDL derived from the M2CDL model for both multi-modal image restoration (MIR) and multi-modal image fusion (MIF) tasks. The network architecture of Deep M2CDL fully matches the optimization steps of the M2CDL model, which makes each network module with good interpretability. Different from handcrafted priors, both the dictionary and sparse feature priors are learned through the network. The performance of the proposed Deep M2CDL is evaluated on a wide variety of MIR and MIF tasks, which shows the superiority of it over many state-of-the-art methods both quantitatively and qualitatively. In addition, we also visualize the multi-modal sparse features and dictionary filters learned from the network, which demonstrates the good interpretability of the Deep M2CDL network.

Principal Uncertainty Quantification With Spatial Correlation for Image Restoration Problems.

Uncertainty quantification for inverse problems in imaging has drawn much attention lately. Existing approaches towards this task define uncertainty regions based on probable values per pixel, while ignoring spatial correlations within the image, resulting in an exaggerated volume of uncertainty. In this paper, we propose PUQ (Principal Uncertainty Quantification) - a novel definition and corresponding analysis of uncertainty regions that takes into account spatial relationships within the image, thus providing reduced volume regions. Using recent advancements in generative models, we derive uncertainty intervals around principal components of the empirical posterior distribution, forming an ambiguity region that guarantees the inclusion of true unseen values with a user-defined confidence probability. To improve computational efficiency and interpretability, we also guarantee the recovery of true unseen values using only a few principal directions, resulting in more informative uncertainty regions. Our approach is verified through experiments on image colorization, super-resolution, and inpainting; its effectiveness is shown through comparison to baseline methods, demonstrating significantly tighter uncertainty regions.

Gaussian filter and fuzzy filter based historical document with geodesic morphology

The technique of image restoration involves recovering an image from a damaged state, typically a distorted and noisy image. In image processing image restoration is a core issues. This paper based on historical documents and these are discovered by archaeologist. Important information about the tradition, religion science, literature. In this Paper, we will use the geodesic morphological model with the fuzzy filter for the purpose of historical document restoration and compare them on the basis of various performance parameters.

Enhancing image denoising: A novel non-local anisotropic diffusion framework based on Caputo derivatives and Gaussian convolution for the Perona–Malik model

In the field of image restoration, denoising is considered one of the most important techniques. It is a pre-processing approach aims to refine image clarity and enhance its overall quality by effectively reducing noise present in the image. The aim is to obtain good-quality images from a version degraded by additive noise or convolutional noise that introduces blur. As a result, more advanced treatments can be performed on the resulting image. In order to remove Gaussian noise from input images, we propose the following methodology using a fractional differential equation in time-space based on Gaussian convolution, where the integer and fractional order derivatives of Caputo can be discretized using finite difference and L1−approximations. Once the equation is solved numerically, the scheme is applied to grayscale digital images using the presented algorithm. The parameters must be optimized and adjusted. As a result of testing with natural images, we are able to successfully suppress the noise present in the images. Aside from that. Our model demonstrates strong visual quality, as verified by the calculation of indexes such as Peak Signal-To-Noise Ratio (PSNR) and Structural Similarity Index Measure (SSIM). Our denoising technique demonstrates its effectiveness in mitigating noise present in both MRI and X-ray images, resulting in improved image quality and diagnostic accuracy for these vital medical imaging modalities.

Terahertz deep learning fusion computed tomography

Terahertz (THz) tomographic imaging based on time-resolved THz signals has raised significant attention due to its non-invasive, non-destructive, non-ionizing, material-classification, and ultrafast-frame-rate nature for object exploration and inspection. However, the material and geometric information of the tested objects is inherently embedded in the highly distorted THz time-domain signals, leading to substantial computational complexity and the necessity for intricate multi-physics models to extract the desired information. To address this challenge, we present a THz multi-dimensional tomographic framework and multi-scale spatio-spectral fusion Unet (MS3-Unet), capable of fusing and collaborating the THz signals across diverse signal domains. MS3-Unet employs multi-scale branches to extract spatio-spectral features, which are subsequently processed through element-wise adaptive filters and fused to achieve high-quality THz image restoration. Evaluated by geometry-variant objects, MS3-Unet outperforms other peer methods in PSNR and SSIM. In addition to the superior performance, the proposed framework additionally provides high scalable, adjustable, and accessible interface to collaborate with different user-defined models or methods.